Rate-distortion optimized video streaming with adaptive playout

We propose a new scheme for streaming media systems that combines Adaptive Media Playout (AMP) with ratedistortion optimized packet transmission. AMP, the clientcontrolled, adaptive modification of the media playout rate, allows us to flexibly adjust the playout deadlines of individual packets, and can therefore reduce reconstruction distortion. This added flexibility incurs a subjective cost, however. In this work we introduce functions that assess the subjective cost of a schedule of playout rate modifications, and we show how to optimize the schedule with respect to these costs and distortion. Because the optimal playout rate schedule and the R-D optimal transmission schedule are interdependent, we solve the two problems jointly. In simulations that model a receiver-driven scenario, results for a short media clip show a more than 2 dB improvement in mean PSNR for R-D optimal transmission scheduling combined with a moderate amount of AMP, over R-D optimal transmission scheduling alone.